test(conformance): vendored scenarios + golden harness, close out S2#26
Merged
Conversation
Lock Studio's detection to the toolkit's over the 15 shared scenarios — the S2 exit gate — and close out the milestone. - src/ocpp/conformance/fixtures/*.json — the 15 shared scenario traces, exported from the toolkit. - src/ocpp/conformance/goldens/*.json — each scenario's expected de-duplicated, sorted FailureCode set, generated from the toolkit's own evaluateScenario (source of truth), tagged contract-v1. - src/ocpp/conformance/harness.zig — a `native test` gate that runs the full engine (parse -> correlate -> detect) over every fixture and asserts its detected code set equals the golden, the same comparison the toolkit's evaluateScenario makes. 15/15 scenarios match. - README documents the contract and its regeneration. The vendored contract lives under src/ so the zero-config build can embed it (ADR-0004). Marks S2 complete in ROADMAP.md and CURRENT_STATE.md. Closes #22
sepehr-safari
added
type:test
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Final slice of S2 — Detection + conformance: the conformance harness that
pins Studio's detection to the toolkit's, and the S2 close-out. This is the
milestone's exit gate — the shared scenarios must match, code-set for code-set.
Changes
src/ocpp/conformance/fixtures/*.json— the 15 shared scenario traces,exported from the toolkit.
src/ocpp/conformance/goldens/*.json— each scenario's expectedde-duplicated, sorted
FailureCodeset, generated from the toolkit's ownevaluateScenario(source of truth, not hand-authored), taggedcontract-v1.src/ocpp/conformance/harness.zig— anative testgate: for each fixture,run the full engine (
parseTrace → buildSessionTimeline → detectFailures),dedupe + sort the codes, and assert equality with the golden — the same
comparison the toolkit's
evaluateScenariomakes. A missing/extra code failsthe scenario by name.
README.md— documents the contract, regeneration, and versioning.Result — S2 exit criterion met
15/15 scenarios match the
contract-v1goldens. Studio's independent Zigimplementation now produces the same failure-code sets as the TypeScript toolkit
across every shared scenario. Verified the gate is real: tampering a golden fails
the test with a named diff.
A note on layout
The plan placed
conformance/at the repo root, but the Native SDK zero-configbuild embeds test data with
@embedFile, which resolves within the source moduleroot (ADR-0004) — a repo-root import fails the module-root boundary. The vendored
contract therefore lives under
src/ocpp/conformance/, keeping the harness fullyhermetic (no runtime file I/O).
Testing
native test -Dplatform=null→ 50/50 pass, including the 15-scenarioconformance gate.
native check --strictandzig fmt --checkclean.Closes #22